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ABSTRACT 


Research  on  the  use  of  virtual  simulation  to  train  Soldiers  and  leaders  in  small  dismounted  units  has  largely 
focused  on  the  use  of  specially  developed,  relatively  high-fidelity  PC-based  simulators.  It  has  been  successful 
in  demonstrating  that  virtual  simulation  can  adequately  support  the  performance  of  a  variety  of  Soldier 
activities,  and  is  perceived  to  be  effective  for  training  both  individual  and  collective  Soldier  skills.  However, 
as  computer  graphics  technology  has  advanced,  the  interface  devices  (head-mounted  or  projection  displays, 
position  trackers,  and  instrumented  mock  weapons)  required  for  immersive  virtual  simulations  have  become  a 
relatively  larger  contributor  to  the  cost  of  simulators  than  the  simulation  engine.  This  raises  the  question  of 
whether  a  high-fidelity  interface  contributes  sufficiently  to  training  effectiveness  to  justify  its  cost.  In  addition, 
the  widespread  availability  and  use  of  video  and  computer  games  has  raised  the  question  of  whether  either 
commercial  games  or  specially  designed  games  can  meet  some  part  of  the  Army ’s  training  needs.  This  paper 
describes  and  discusses  these  issues  in  detail,  presents  supporting  research  evidence,  and  describes  future 
research  needs. 

1.0  INTRODUCTION 

The  potential  benefits  of  using  virtual  simulation  to  train  Soldiers  and  leaders  in  small  dismounted  units  have 
been  recognized  for  more  than  a  decade.  During  that  period  the  U.S.  Army  Research  Institute  for  the 
Behavioral  and  Social  Sciences  (ARI)  has  conducted  research  to  evaluate  the  capabilities  of  virtual  technology 
to  provide  dismounted  Soldier  and  leader  training,  the  effectiveness  of  that  training  and  instructional 
techniques  to  enhance  it,  and  mechanisms  for  providing  performance  measurement  and  feedback.  The  focus 
of  the  research  has  largely  been  on  the  use  of  specially  developed,  relatively  high-fidelity  PC-based  virtual 
simulators.  Our  approach  has  included  both  field  assessments  of  virtual  training  technologies  with  Soldiers  in 
realistic  training  settings  and  more  scientifically  rigorous  experimentation  in  a  laboratory  setting.  The  research 
program  has  been  successful  in  demonstrating  that  virtual  simulation  can  adequately  support  the  performance 
of  a  variety  of  Soldier  activities,  and  is  perceived  to  be  effective  for  training  both  individual  and  collective 
Soldier  skills. 
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During  the  past  few  years,  however,  there  have  been  changes  in  both  the  training  needs  of  the  Army  and  the 
range  of  potential  training  solutions  available.  These  changes  present  both  new  challenges  and  new 
opportunities. 

In  part  because  of  changes  in  doctrine  describing  how  we  expect  to  fight  in  the  future,  and  in  part  because  of 
the  lessons  we  have  learned  in  current  operations  in  Iraq  and  Afghanistan,  the  Army  now  recognizes  a  need  to 
train  Soldiers  and  leaders  to  be  adaptable,  capable  of  responding  to  rapidly  changing  situations,  and  attuned  to 
cultural  conditions,  in  addition  to  being  proficient  in  high-intensity  combat  operations.  With  this  new  context, 
it  is  proposed  that  virtual  simulation  could  be  used  not  only  to  train  for  relatively  straight-forward  combat 
operations,  but  also  to  train  the  decision  making  and  command  and  control  skills  that  small  unit  leaders  and 
Soldiers  need  in  the  current  operating  environment.  This  change  in  training  needs  has  implications  for 
simulation  requirements. 

At  the  same  time,  advances  in  computer  graphics  technology  and  processing  power  have  made  it  possible  to 
generate  the  graphics  and  calculate  weapons  and  environmental  effects  with  moderate-cost  PCs,  at  costs 
approaching  1%  of  the  cost  of  the  graphics  generators  of  a  decade  ago.  However,  the  costs  of  the  interface 
devices  (head-mounted  or  projection  displays,  position  trackers,  and  instrumented  mock  weapons)  have 
declined  only  slightly  during  the  same  period,  making  them  a  much  larger  contributor  to  the  cost  of  simulators 
than  the  computer  itself.  Do  these  high-fidelity  interface  devices  (locomotion  devices  or  immersive  displays) 
contribute  enough  to  training  effectiveness  to  justify  their  cost? 

In  addition,  the  availability  and  presumed  widespread  use  of  video  and  computer  games  by  Soldiers  has 
created  interest  in  the  use  of  either  commercial  or  specially  designed  games  to  meet  some  part  of  the  Army’s 
training  needs.  While  such  games  were  initially  seen  as  being  too  constrained  and  insufficiently  realistic  to 
provide  training  benefits,  the  potential  to  take  advantage  of  both  the  huge  commercial  investment  in  video 
game  technology  and  the  experience  of  Soldiers  playing  video  games  has  tremendous  appeal.  Are  Soldiers  as 
familiar  with  these  games  as  we  assume,  and  how  well  do  the  games  train? 

This  paper  will  describe  and  discuss  these  issues  in  more  detail,  presents  supporting  research  evidence  from 
recent  field  assessments  and  experiments,  and  describe  future  research  needs. 


2.0  BACKGROUND  RESEARCH 

ARI  has  been  interested  in  the  use  of  virtual  simulation  to  train  dismounted  Soldiers  for  more  than  a  decade. 
Our  research  began  with  analyses  of  the  feasibility  of  using  VE  technology  for  dismounted  Soldier  training 
and  the  identification  of  difficult  technical  problems  and  research  issues  (Levison  and  Pew,  1993;  Jacobs  et 
al.,  1994).  With  these  as  a  basis,  we  initiated  a  laboratory  research  program  to  investigate  critical  behavioral 
science  research  issues,  ranging  from  the  investigation  of  interface  effects  on  task  performance  to  the  effects 
of  geographically  distributed  team  members  on  training  effectiveness.  Reviews  of  the  research  are  contained 
in  Knerr,  Lampton,  Singer,  Witmer,  and  Goldberg  (1998)  and  Lampton,  Knerr,  Martin,  and  Washburn  (2002). 

In  1997  we  began  to  expand  our  research  program  to  include  field  experiments  as  well  as  laboratory 
experimentation.  We  were  able  to  do  this  by  forming  partnerships  with  other  Army  research  and 
developments  organizations  with  which  we  had  goals  in  common.  While  these  field  experiments  sometimes 
lacked  the  control  and  precision  of  our  laboratory  experiments,  they  compensated  by  providing  an  opportunity 
to  obtain  data  on  the  use  of  virtual  simulation  technology  by  Soldiers  as  they  performed  actual  Soldier  tasks. 
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Our  involvement  with  field  experiments  began  with  Dismounted  Warrior  Network  (DWN).  DWN  was  a  U.S. 
Army  Simulation,  Training,  and  Instrumentation  Command  (STRICOM)1  research  and  development  program 
to  develop  a  virtual  simulation  capability  for  dismounted  Soldiers.  A  series  of  experiments  was  conducted 
during  1997  to  investigate  different  simulator  interfaces  (Lockheed  Martin  Corporation,  1997;  Pleban,  Dyer, 
Salter,  and  Brown,  1998).  A  follow-on  project  entitled  DWN  Enhancements  for  Restricted  Terrain  (DWN 
ERT)  focused  on  Military  Operations  in  Urban  Terrain  (MOUT).  Simulators  were  modified  and  capabilities 
added  based  on  lessons  learned  in  the  DWN  experiments.  Experiments  conducted  in  1998  produced  the 
general  conclusion  that  although  the  simulation  capability  was  still  not  sufficiently  mature  to  support  the 
training  of  squad  missions  in  an  urban  environment,  it  showed  potential  (Lockheed  Martin  Corporation,  1998; 
Salter,  Eakin,  and  Knerr,  1999). 

During  the  next  phase  of  the  research,  ARI  participated  in  a  multi-organizational  program  entitled  “Virtual 
Environments  for  Dismounted  Soldier  Simulation,  Training  and  Mission  Rehearsal.”  This  four-year  effort 
focused  on  overcoming  critical  technological  challenges  to  high  fidelity  dismounted  Soldier  simulation 
identified  during  the  previous  research.  These  critical  challenges  included:  simulating  locomotion;  tracking 
weapons  and  body  positions;  creating  realistic  performance  of  computer-controlled  dismounted  friendly  and 
enemy  Soldiers;  simulation  of  night  equipment  and  sensor  images;  making  terrain  and  structures  dynamic; 
developing  appropriate  training  strategies  and  methods;  assessing  individual  and  unit  performance;  and 
determining  transfer  of  training  from  virtual  to  live  environments.  At  the  end  of  each  year  of  the  effort,  we 
conducted  a  series  of  exercises  with  Soldiers  to  help  assess  our  progress.  The  overall  effort  was  successful, 
although  some  of  the  individual  technologies  were  identified  as  requiring  further  improvement  or  not  yet 
sufficiently  mature  for  use.  One  of  the  recommendations  resulting  from  the  effort  was  the  following: 

Given  the  current  state  of  technology,  it  appears  that  VE  could  be  used  effectively  for  some 
types  of  training  and  some  stages  of  training.  VE  could  be  used  for  the  walk  phase  of  the 
training,  concentrating  on  improving  the  decision  making,  situation  awareness, 
communication,  and  coordination  skills,  while  real  world  training  could  place  greater 
emphasis  on  the  motor  skills.  Therefore,  although  there  are  still  further  improvements  that  can 
be  made  in  the  individual  technologies,  as  identified  earlier  in  this  report,  the  next  step  should 
be  an  advanced  development  effort,  taking  a  total  systems  approach,  to  produce  a  prototype 
VE  training  system  for  the  leaders  of  small  dismounted  Infantry  units.  (Knerr  et  al,  2003,  pp. 

47-48). 

3.0  THE  VIRTUAL  INTEGRATED  MOUT  TRAINING  SYSTEM  (V-IMTS) 

Partly  as  a  result  of  this  recommendation,  the  U.S.  Army  Research,  Development  and  Engineering  Command 
Simulation  and  Training  Technology  Center  (RDECOM  STTC)  led,  and  ARI  participated  in,  the  fielding  and 
evaluation  of  a  prototype  deployable  dismounted  Solder  simulation  system,  the  Virtual  Integrated  MOUT 
Training  System  (V-IMTS).  This  will  be  described  in  detail  because  in  many  ways  it  represents  the 
culmination  of  more  than  a  decade  of  effort,  and  because  the  results  provide  the  basis  for  much  of  the 
discussion  which  follows. 

A  deployable  shelter  approximately  40  feet  square  was  placed  next  to  the  control  center  of  the  Cassidy 
Combined  Arms  Collective  Training  Center,  Fort  Campbell,  KY.  The  center  consists  of  a  28-building 
complex  of  one-  to  four-story  buildings  representing  a  small  town.  The  control  center  is  adjacent  to  but 


1  As  a  result  of  a  re-organization  on  1  October  2002,  the  participating  element  within  STRICOM  became  a  part  of  the  Research, 
Development,  and  Engineering  Command,  Simulation  and  Training  Technology  Center. 
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physically  separated  from  the  complex.  Three  Soldier  Visualization  Station  Immersive  (SVSI)  simulators  and 
six  SVS  Desktop  (SVSD)  simulators  were  installed  inside  the  shelter  as  shown  in  Figure  1.  The  SVSI  is  an 
immersive  virtual  simulator  (Figure  2)  which  uses  a  rear-screen  projection  system  to  present  images  (800  X 
600  resolution)  on  a  screen  approximately  10  feet  wide  by  7.5  feet  high.  The  Soldier’s  head  and  weapon  are 
tracked  using  an  acoustic/inertial  tracking  system.  The  Soldier  navigates  through  the  environment  via  a  thumb 
switch  located  on  the  weapon.  The  SVSD  is  functionally  similar  to  the  SVSI,  but  the  Soldier  sits  at  a  PC  and 
views  the  simulation  on  an  LCD  monitor.  A  joystick  is  used  to  control  movement  and  weapons  use.  The  squad 
leader  and  two  fire  team  leaders  used  the  SVSIs.  The  remaining  Soldiers  used  the  SVSDs  set  on  tables  located 
immediately  behind  their  fire  team  leader.  The  SVSI  and  SVSD  simulators  were  included  in  a  network  with 
computer-generated  forces,  an  After  Action  Review  (AAR)  system,  and  simulators  used  for  human  opposing 
forces.  All  Soldiers  on  the  network  could  see  and  interact  with  each  other’s  avatars.  Voice  communication 
among  the  Soldiers  was  unaided  (i.e.,  they  shouted)  except  for  a  handheld  radio  used  by  the  squad  leader  to 
communicate  with  his  platoon  leader. 
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Figure  1:  A  Diagram  of  the  V-IMTS  Deployable  Shelter  and  Simulation  Network  Components. 


Figure  2:  A  Soldier  in  the  SVSI. 
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Twenty-seven  Soldiers  from  three  existing  squads  participated  in  the  assessment.  While  68%  of  the  Soldiers 
had  served  a  tour  in  Iraq  or  Afghanistan,  all  three  squad  leaders,  and  three  of  the  six  team  leaders,  had  served 
in  their  current  duty  position  for  one  month  or  less.  Their  unit  personnel,  assisted  by  technical  personnel, 
controlled  the  exercises  and  conducted  the  AARs.  The  scenarios  consisted  of  multiple  variants  of  two 
missions:  Cordon  and  Search  a  Building  and  Attack/Assault  a  Building.  The  variants  differed  in  the  buildings 
involved  and  the  positions  and  actions  of  the  enemy. 

It  was  intended  that  each  squad  would  conduct  two  live  exercises  separated  by  up  to  six  virtual  exercises 
(depending  on  time  requirements)  over  a  two-day  period.  However,  because  of  severe  weather,  only  Squad  1 
was  trained  exactly  this  way.  Each  squad  completed  two  live  exercises  separated  by  two,  three,  or  six  virtual 
exercises.  Overall  mean  duration  of  the  virtual  exercises  was  16.7  minutes.  Squad  1  participated  in  virtual 
exercises  for  96  minutes,  squad  2  participated  for  77  minutes,  and  squad  3  participated  for  44  minutes. 

At  the  completion  of  their  training,  Soldiers  completed  a  Simulator  Capability  Questionnaire,  which  asked 
them  to  rate  how  well  they  could  perform  each  of  54  activities  in  the  simulators,  and  a  Training  Effectiveness 
Questionnaire ,  on  which  they  rated  their  improvement  on  each  of  1 1  tasks  as  a  result  of  their  training. 

The  activities  which  Soldiers  indicated  could  be  performed  best  included  outdoor  movement,  identification  of 
types  of  people  (civilians,  non-combatants  within  a  room,  enemy  soldiers),  identification  of  tactically 
significant  areas  (sectors  of  observation  and  responsibility),  and  individual  weapons  use  (but  not  grenades). 
Activities  which  Soldiers  said  they  could  not  perform  well  included  maneuver  indoors  (close  to  others,  past 
furniture,  close  to  walls,  around  objects,  past  other  personnel,  around  corners,  through  doorways,  up  and  down 
stairs),  and  identifying  the  source  and  type  of  fire  (enemy  or  friendly),  either  by  auditory  or  visual  cues. 
Ratings  by  the  leaders  using  the  SVSI  and  the  Soldiers  using  the  SVSD  were  similar  but  not  identical.  The 
SVSD  was  rated  slightly  but  not  significantly  higher  than  the  SVSI  (mean  rating  of  1.76  vs.  1.66  on  a  scale  of 
0  -  3).  The  SVSI  and  SVSD  ratings  were  positively  correlated  (r  =  .62),  indicating  that  in  general  the  same 
activities  tended  to  be  rated  similarly  on  both  simulators.  However,  there  were  exceptions.  The  SVSI  was 
rated  significantly  higher  (p<.05)  than  the  SVSD  on  the  activities  Fire  weapon  accurately  and  Aim  weapon , 
while  the  opposite  was  true  for  Identify  assigned  sectors  of  observation  and  Identify  enemy  soldiers. 

The  training  effectiveness  ratings  are  presented  in  Figure  3.  The  mean  rating  was  1.8  on  a  scale  where  1.0 
equalled  slight  improvement  and  2.0  equalled  moderate  improvement.  Interestingly,  both  the  leaders  and  the 
Soldiers  reported  about  the  same  amount  of  overall  improvement,  although  the  skills  on  which  they  reported 
improving  the  most  differed.  Leaders  reported  the  greatest  improvement  on  Control  of  squad/fire  team 
movement  during  the  assault ,  Assess  the  tactical  situation ,  Plan  a  tactical  operation ,  and  Coordinate  activities 
with  your  chain  of  command.  Soldiers  reported  the  most  improvement  on  Plan  a  tactical  operation , 
Coordinate  activities  with  your  chain  of  command ,  and  Communicate  with  members  of  your  team  or  squad. 
Perhaps  because  of  the  small  numbers  involved,  the  differences  between  leader  and  Soldier  ratings 
approached  significance  for  only  one  task,  Control  of  squad/fire  team  movement  during  the  assault. 


4.0  THE  COSTS  OF  FULLY  IMMERSIVE  SIMULATION 

An  interesting  and  unexpected  aspect  of  these  results  is  the  lack  of  a  significant  difference  in  rated  training 
effectiveness  between  the  Soldiers  trained  in  the  SVSD  and  the  leaders  trained  in  the  SVSI.  Admittedly,  they 
were  performing  and  learning  different  tasks,  but  nevertheless  it  seems  to  indicate  that  it  might  be  productive 
to  investigate  whether  the  features  of  fully  immersive  simulators  produce  a  training  benefit  that  justifies  the 
cost  premium  over  a  simple  desktop  computer. 
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The  cost  difference  is  substantial.  Table  1,  based  on  cost  data  provided  by  Loftin,  Dryer,  Belfore,  Petty, 
Phillips,  Garcia,  Seevi,  Lusso,  Mastaglio,  and  Park  (2004),  shows  the  estimated  cost  of  the  hardware 
components  of  an  immersive  dismounted  Soldier  simulator  built  to  the  specifications  of  the  Army’s  proposed 
Soldier  CATT  Virtual  Warrior  station.  These  estimates  were  made  in  May  2004.  Note  that  they  do  not  include 
the  computer,  simulation  software,  or  a  voice  communications  system.  Even  using  the  low  cost  estimate  for 
each  component  with  an  HMD,  instead  of  the  more  expensive  projection  display,  per-system  cost  is  $81,000. 
In  contrast,  the  Dell  line  of  gaming  PCs  begins  at  $1000,  and  a  very  high  end  Dell  gaming  PC  (XPS  600)  with 
a  24”  LCD  monitor  and  joystick  costs  less  than  $4200. 

Table  1:  Immersive  Simulator  Cost  Estimates 


Component 

Low  Estimate 

High  Estimate 

Tracking  System 

$43,000 

$50,000 

HMD  (1280X1024  resolution)) 

$26,000 

$40,000 

Projection  Display 

$30,000 

$350,000 

Targeting/ Augmented  Display 

$5,000 

$8,000 

Binoculars/Nigh  Vision  Goggles 

$4,000 

$25,000 

Sound  System 

$3,000 

$8,000 

Total  with  Projection  Display 

$85,000 

$441,000 

Total  with  HMD 

$81,000 

$131,000 

21  -6 


RTO-MP-HFM-136 


dMNATO 
WP  I  OTAN 


Current  Issues  in  the  Use  of 
Virtual  Simulations  for  Dismounted  Soldier  Training 


5.0  DOES  FULL  IMMERSION  FACILITATE  TRAINING  AND  TRANSFER? 

This  cost  differential  makes  it  prudent  to  ask  whether  the  training  benefit  obtained  by  using  a  high-fidelity 
immersive  interface  provides  sufficient  training  benefit  to  justify  the  additional  cost.  Loftin,  Scerbo, 
McKenzie,  Catanzaro,  Bailey,  Phillips,  and  Perry  (2004)  have  come  closest  to  addressing  this  issue  directly. 
They  compared  a  monitor  and  CAVE  for  training  the  conduct  of  checkpoint  operations.  Because  of  radically 
unequal  numbers  of  participants  in  the  monitor  and  CAVE  groups,  they  ran  no  statistical  tests,  but  reported 
that  the  trainees  in  the  fully  immersive  system  consistently  performed  better.  However,  they  concluded  that 

Although  this  difference  was  found  across  groups  and  conditions,  the  magnitude  of  the 
difference  was  not  dramatic.  Thus,  the  ability  to  port  a  similar  training  experience  to  a  less 
expensive  PC  platform  without  major  performance  differences  underscores  the  potential  for 
providing  greater  access  to  this  type  of  VE  training  in  a  much  more  cost  effective  medium,  (p. 

12). 

Other  research  addressing  the  question  of  interface  fidelity  on  training  effectiveness  is  limited,  and  has  usually 
involved  a  comparison  of  VE  systems  with  and  without  head-coupled  visual  displays. 

Grant  and  Magee  (1997)  used  a  walking  interface  in  an  experiment  investigating  the  acquisition  and  transfer 
of  spatial  knowledge.  The  participants'  task  was  to  learn  the  location  of  a  number  of  landmark  objects  in  a 
science  museum.  During  a  transfer  test  in  the  actual  building,  the  walking  simulator  group  walked 
significantly  less  distance  to  find  all  of  the  landmarks  than  did  the  joystick  trained  group,  but  they  did  not  find 
them  significantly  faster.  Grant  and  Magee  claim  this  as  evidence  that  there  is  something  learned  from  the 
walking  interface  in  addition  to  the  visual  information  that  can  be  used  for  spatial  navigation. 

Singer,  Allen,  McDonald,  and  Gildea  (1997)  found  that  training  a  Hi-VE  interface  (head-linked  stereoscopic 
HMD  and  treadmill)  produced  better  configuration  knowledge  and  more  rapid  response  time  than  a  Low-VE 
interface  (non-head-linked  stereoscopic  HMD  and  joystick)  in  a  terrain  learning  task. 

Jacquet  (2002)  found  only  a  minor  difference  between  groups  trained  to  perform  a  maintenance  task  in  VR  vs. 
a  desktop  simulator. 

In  summary,  it  appears  that  head-tracked  visual  displays  make  a  difference  in  performing  spatially-oriented 
tasks  and  acquiring  spatial  knowledge,  but  this  difference,  based  on  a  limited  number  of  experiments,  does  not 
appear  to  be  large. 


6.0  DO  SOLDIERS  PLAY  GAMES? 

The  use  of  commercial  games  connected  via  local  networks  or  the  internet  is  also  being  proposed  as  a  training 
solution.  The  attractiveness  of  these  games,  in  addition  to  presumed  training  effectiveness  (see  below),  is  that 
Soldiers  are  assumed  to  already  be  familiar  with  them,  and  motivated  to  train  with  them,  even  in  off-duty 
hours.  Are  these  assumptions  justified? 

Roberts,  Foehr,  and  Rideout  (2005)  conducted  an  extensive  survey  of  American  youth  and  media  in  2004. 
They  found  that  8-18  year  olds  spent  an  average  of  19  minutes  per  day  playing  PC  games  and  49  minutes 
playing  console  and  handheld  games.  Fifty-nine  percent  play  some  type  of  interactive  games  on  any  given  day 
(68%  of  males  and  51%  of  females).  These  results  seem  to  be  fairly  consistent  across  race  and  parent’s 
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education  and  income.  However,  it  cannot  be  determined  from  the  report  what  percentage  of  youth  rarely  or 
never  play  interactive  games. 

No  recent  large-scale  surveys  of  Soldier  game  playing  have  been  conducted.  However,  Soldiers  participating 
in  evaluations  of  simulations  and  PC  games  have  been  asked  about  their  game -playing  experience.  Soldiers  in 
the  V-IMTS  evaluation  described  earlier  reported  a  mean  of  9.5  hours  (median  6.5  hours)  per  week  playing 
computer  or  video  games.  A  sample  of  27  Infantry  Soldiers  from  a  separate  assessment  conducted  in  FY  2004 
(Knerr,  Garrity,  and  Lamp  ton,  2004)  reported  a  mean  of  8.5  hours  (median  5.0  hours)  per  week  playing 
computer  or  video  games.  However,  7%  and  22%  of  the  Soldiers  in  these  samples,  respectively,  did  not  play 
video  games.  Beal  and  Christ  (2005)  found  that  30%  of  their  sample  of  39  Infantry  lieutenants  involved  in  the 
evaluation  of  the  Rapid  Decision  Trainer  did  not  play  video  games,  and  51%  described  themselves  as  Novice 
PC  game  players.  Only  13%  reported  playing  more  than  two  hours  per  week.  In  their  evaluation  of  the  Full 
Spectrum  Command  (FSC)  game,  Beal  and  Christ  (2004)  found  that  57%  of  their  sample  of  54  Infantry 
lieutenants  and  captains  did  not  play  computer  games.  Diller,  Roberts,  Blankenship,  and  Nielsen  (2004),  when 
evaluating  the  game  DARWARS  Ambush!,  found  that  14  of  their  18  participants  reported  playing  computer 
games  for  at  least  one  hour  per  week  (mean  =  12.9  hours  per  week).  Their  sample  included  2  officers  and  16 
enlisted  personnel,  three  of  whom  were  women. 

The  results  involving  enlisted  personnel  seem  fairly  consistent  with  those  of  the  Roberts,  Foehr,  and  Rideout 
(2005)  survey  data  reported  above.  The  results  involving  junior  officers  seem  very  different.  Whether  this  is  a 
result  of  differences  in  age,  education,  social  factors,  or  the  particular  questions  asked  is  not  clear.  It  is  safe  to 
assume  that  a  large  portion  of  our  potential  trainees  are  familiar  with  computer  or  video  games.  However,  we 
cannot  assume  that  all  of  them  are  skilled  or  enthusiastic  game  players.  This  does  not  mean  that  games  could 
not  provide  effective  training  for  them,  but  it  does  suggest  a  need  for  adequate  training  in  playing  the  game 
before  training  begins  in  earnest. 

A  related  question  is  whether  Soldiers  have  access  to  computers  to  use  for  training  in  their  off-duty  hours.  The 
ARI  Army  Personnel  Survey  Office  conducted  a  survey  in  the  fall  of  2002  and  found  that  99.1%  of  all  Army 
officers  and  87.4%  of  all  enlisted  personnel  reported  having  access  to  the  internet,  while  98.7%  and  84.1%, 
respectively,  reported  having  access  to  a  PC.  The  group  with  the  lowest  access  was  junior  enlisted  personnel, 
with  only  80.4%  having  access  to  the  internet  and  74.8%  having  access  to  a  PC.  These  data  may  have 
changed,  but  no  recent  survey  data  are  available.  Moreover,  having  a  computer  and  internet  access  in  the 
barracks  or  residence  does  not  guarantee  that  those  resources  can  be  used  by  the  Soldier  for  training  in  off- 
duty  hours. 


7.0  CAN  GAMES  PROVIDE  EFFECTIVE  TRAINING? 

The  most  important  question  about  games  is  whether  they  provide  effective  training.  While  most  researchers 
seem  to  agree  that  computer  games  can  be  effective  training  tools,  there  is  very  little  empirical  evidence  to 
support  that  conclusion  (Belanich,  Mullen,  and  Dressel,  2004).  In  a  recent  comprehensive  review  of  the 
effectiveness  of  instructional  games,  Hayes  (2005)  concluded  that  “although  research  has  shown  that  some 
games  can  provide  effective  learning  for  a  variety  of  learners  for  several  different  tasks  (e.g.,  math,  attitudes, 
electronics,  and  economics),  this  does  not  tell  us  whether  to  use  a  game  for  our  specific  instructional  task”  (p. 
6).  Likewise,  Bonk  and  Dennen  (2005)  noted  the  lack  of  research  on  the  impact  of  instructional  games  on 
analysis  and  decision  making  skills.  In  particular,  there  is  little  research  that  would  particularly  apply  to 
training  dismounted  Soldier  unit  and  Leader  skills. 
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Beal  and  Christ  (2004)  conducted  an  evaluation  of  FSC,  a  PC  game  designed  to  let  prospective  Infantry 
Company  Commanders  analyze,  plan  and  execute  missions.  While  the  students  believed  that  FSC  was  an 
effective  means  of  training,  this  was  not  supported  by  other  data,  quite  possibly  because  the  students  had 
limited  training  time  with  the  game.  Beal  and  Christ  (2005)  also  conducted  an  evaluation  of  the  Rapid 
Decision  Trainer  (RDT),  a  PC-based  simulation  developed  to  prepare  Infantry  lieutenants  to  serve  as  rifle 
platoon  leaders  during  a  live  fire  exercise.  The  trainees  reported  that  they  believed  that  the  RDT  had  training 
value.  Diller,  Roberts,  Blankenship,  and  Nielsen  (2004)  reported  that  their  trainees  gave  the  game 
DARWARS  Ambush!  (a  PC  game  designed  to  train  convoy  operations)  high  ratings  of  effectiveness.  No 
performance  data  were  obtained. 

Evidence  about  the  effectiveness  of  PC  games  for  training  Soldier  skills,  therefore,  is  based  on  the  subjective 
opinions  of  trainees,  and  not  on  objective  measures  of  training  effectiveness. 


8.0  DISCUSSION 

We  would  like  to  be  able  to  make  recommendations  to  the  Army  about  the  appropriate  roles  of  immersive 
simulations,  low  cost  simulations,  and  PC-based  games  for  training  dismounted  Infantry  Soldiers,  leaders,  and 
small  units.  These  approaches  have  different  acquisition,  operation,  and  support  costs.  There  is  a  lack  of  sound 
empirical  evidence,  based  on  objective  measures  of  performance,  which  would  lead  to  recommendations 
about  training  and  cost  effectiveness  in  which  we  could  have  confidence.  Researchers  and  research  reviewers 
seem  to  agree  that  each  of  these  approaches  can  be  effective  if  used  appropriately.  Research  to  date  has  largely 
addressed  the  question  of  whether  one  of  those  approaches  can  be  effective,  and  (with  few  exceptions)  has  not 
generally  compared  the  effectiveness  or  cost-effectiveness  of  the  different  approaches.  A  likely  reason  for  this 
is  that  the  addressing  this  issue  experimentally  would  require  a  substantial  investment  in  equipment,  support 
personnel,  and  Soldier  trainees.  Yet  without  this  evidence,  the  Army  does  not  have  a  sound  basis  for  selecting 
the  most  cost  effective  mix  of  training  methods. 

In  addition  to  the  question  of  effectiveness,  we  would  also  like  to  know  more  about  the  motivation  of 
prospective  trainees  to  use  computer  or  video  games  and  their  familiarity  and  proficiency  with  them.  Data 
from  small  samples  of  Soldiers  who  participated  in  evaluations  suggest  that  a  fairly  large  percentage  of  male 
enlisted  personnel,  possibly  75%  or  higher,  play  computer  or  video  games  weekly  or  more  frequently. 
Officers  appear  to  play  less  frequently  than  enlisted  Soldiers.  In  addition,  based  on  civilian  data,  we  would 
expect  that  female  Soldiers  would  play  less  frequently  than  male  Soldiers.  We  know  little  about  the  types  of 
games  that  Soldiers  play  (e.g.,  first  person  shooter  vs.  role  player).  Since  game  playing  experience  has 
implications  for  the  types  of  games  that  could  be  used  for  training  and  the  amount  and  type  of  prerequisite 
game  playing  training  required,  we  would  like  to  have  a  much  better  picture  of  the  both  current  and  projected 
Soldier  and  leader  familiarity  and  experience  with  different  types  of  interactive  games. 


9.0  REFERENCES 

Beal,  S.A.  and  Christ,  R.R.  (2004).  Training  Effectiveness  of  the  Full  Spectrum  Command  Game  (TR  1140). 
Arlington,  VA:  U.S.  Army  Research  Institute  for  the  Behavioral  and  Social  Sciences.  (ADA4 19670) 

Beal,  S.A.  and  Christ,  R.R.  (2005).  Soldier  Perceptions  of  the  Rapid  Decision  Trainer  (RR1835).  Arlington, 
VA:  U.S.  Army  Research  Institute  for  the  Behavioral  and  Social  Sciences.  (ADA436993) 


RTO-MP-HFM-1 36 


21  -9 


Current  Issues  in  the  Use  of 

Virtual  Simulations  for  Dismounted  Soldier  Training 


ORGANIZATION 


Belanich,  J.,  Mullen,  L.N.,  and  Dressel,  J.D.  (2004).  Symposium  on  PC-based  Simulation  and  Gaming  for 
Military  Training  (RP  2005-1).  Arlington,  VA:  U.S.  Army  Research  Institute  for  the  Behavioral  and 
Social  Sciences.  (ADA415918) 

Bonk,  C.J.,  and  Dennen,  V.P.  (2005).  Massive  Multiplayer  Online  Gaming:  A  Research  Framework  for 
Military  education  and  Training  (TR  2005-1).  Washington,  DC:  Office  of  the  Under  Secretary  of 
Defense  (Personnel  and  Readiness),  Readiness  and  Training  Directorate.  Advanced  Distributed 
Learning  (ADL)  Initiative.  (ADA431371) 

Diller,  D.E.,  Roberts,  B.,  Blankenship,  S.  &  Nielsen,  D.  (2004,  December).  DARWARS  Ambush!  - 
Authoring  Lessons  Learned  in  a  Training  Game.  Proceedings  of  the  2004  Interservice/Industry 
Training,  Simulation,  and  Education  Conference  (I/ITSEC).  Orlando,  FL. 

Grant,  S.C.,  &  Magee,  L.E.  (1997,  December).  Navigation  in  a  virtual  environment  using  a  walking  interface. 
Proceedings  of  the  NATO  RSG-18  Workshop  on  Capability  of  Virtual  Reality  to  Meet  Military 
Requirements.  Orlando,  FL. 

Hayes,  R.T.  (2005).  The  effectiveness  of  instructional  games:  A  literature  review  and  discussion  (TR  2005- 
004).  Orlando,  FL:  Naval  Air  Warfare  Center  Training  Systems  Division. 

Jacobs,  R.  S.,  Crooks,  W.  H.,  Crooks,  J.  R.,  Colburn,  E.,  Fraser,  II,  R.  E.,  Gorman,  P.  F.,  Madden,  J.  L., 
Furness,  III,  T.  A.,  &  Tice,  S.  E.  (October,  1994).  Behavioral  Requirements  for  Training  and  Rehearsal 
in  Virtual  Environments.  (TR-1011),  Alexandria,  VA:  U.S.  Army  Research  Institute  for  the  Behavioral 
and  Social  Sciences.  (ADA286311) 

Jacquet,  C.R.  (2002).  A  Study  of  the  Effects  of  Virtual  Reality  on  the  Retention  of  Training.  Unpublished 
Master’s  Thesis,  University  of  Central  Florida.  Orlando,  FL.  (ADA401640) 

Knerr,  B.W.,  Garrity,  P.J.,  and  Lampton,  D.R.  (2004,  December).  Embedded  training  for  future  force 
warriors:  An  assessment  of  wearable  virtual  simulators.  Proceedings  of  the  24th  Army  Science 
Conference.  Orlando,  FL.  (ADA433376) 

Knerr,  B.W.,  and  Lampton,  D.R.  (2005).  An  Assessment  of  the  Virtual-Integrated  MOUT  Training  System  (V- 
IMTS)  (TR  1163).  Alexandria,  VA:  U.S.  Army  Research  Institute  for  the  Behavioral  and  Social 
Sciences.  (ADA438351) 

Knerr,  B.  W.,  Lampton,  D.  R.,  Singer,  M.  J.,  Witmer,  B.  G.,  &  Goldberg,  S.  L.  (1998).  Virtual  environments 
for  dismounted  soldier  training  and  performance:  Results,  recommendations,  and  issues  (ARI 
Technical  Report  1089).  Alexandria,  VA:  U.S.  Army  Research  Institute  for  the  Behavioral  and  Social 
Sciences.  (ADA360109) 

Knerr,  B.W.,  Lampton,  D.R.,  Thomas,  M.,  Comer,  B.D.,  Grosse,  J.R..,  Centric,  J.,  Blankenbeckler,  P.,  Dlubac, 
M.,  Wampler,  R.L.,  Siddon,  D.,  Garfield,  K.,  Martin,  G.A.,  &  Washburn,  D.A.  (2003).  Virtual 
environments  for  dismounted  soldier  simulation,  training,  and  mission  rehearsal:  Results  of  the  FY 
2002  culminating  event  (ARI  Technical  Report  1138).  Alexandria,  VA:  U.S.  Army  Research  Institute 
for  the  Behavioral  and  Social  Sciences.  (ADA417360) 

Levison,  W.  H.,  &  Pew,  R.  W.  (1993).  Use  of  virtual  environment  training  technology  for  individual  combat 
simulation.  (Technical  Report  971).  Alexandria,  VA:  U.S.  Army  Research  Institute  for  the  Behavioral 
and  Social  Sciences.  (ADA263546) 


21  -10 


RTO-MP-HFM-136 


dMNATO 
WP  I  OTAN 


Current  Issues  in  the  Use  of 
Virtual  Simulations  for  Dismounted  Soldier  Training 


Lampton,  D.  R.,  Knerr,  B.  W.,  Martin,  G.  A.,  &  Washburn,  D.  A.  (2002).  Trends  in  ten  years  of  immersive 
virtual  environment  research.  Proceedings  of  the  2002  IMAGE  Conference ,  pp.  214  -  222.  Scottsdale, 
AZ:  The  IMAGE  Society. 

Lockheed  Martin  Corporation.  (1997).  Dismounted  warrior  network  front  end  analysis  experiments 
(Advanced  Distributed  Simulation  Technology  II,  Dismounted  Warrior  Network  DO  #0020,  CDRL 
AB06,  ADST-II-CDRL-DWN-9700392A).  Orlando,  FL:  US  Army  Simulation,  Training  and 

Instrumentation  Command.  (ADA344365) 

Lockheed  Martin  Corporation.  (1998).  Dismounted  warrior  network  enhancements  for  restricted  terrain 
(Advanced  Distributed  Simulation  Technology  II,  Dismounted  Warrior  Network  DO  #0055,  CDRL 
AB01,  ADST-II-CDRL-DWN-9800258A).  Orlando,  FL:  US  Army  Simulation,  Training  and 

Instrumentation  Command.  (ADA370504) 

Loftin,  R.B.,  Dryer,  D.,  Belfore,  L.,  Petty,  M.,  Phillips,  M.,  Garcia,  H.,  Seevi,  J.,  Lusso,  A.,  Mastaglio,  T.,  and 
Park,  N.  (2004).  Soldier  CATT front  end  analysis:  Results  of  Technology  Assessment.  Portsmouth,  VA: 
MYMIC  LLC. 

Loftin,  R.B.,  Scerbo,  M.W.,  McKenzie,  R.,  Catanzaro,  J.M.,  Bailey,  N.R.,  Phillips,  M.A.,  and  Perry,  G. 
(2004).  Training  in  peacekeeping  operations  using  virtual  environments.  Paper  presented  at  the  RTO 
HFM  Symposium  on  “ Advanced  Technologies  for  Military  Training.  ”  (ADA428142) 

Pleban,  R.  J.,  Dyer,  J.  L.  Salter,  M.  S.,  &  Brown,  J.  B.  (1998).  Functional  capabilities  of  four  virtual 
individual  combatant  (VIC)  simulator  technologies:  An  independent  assessment  (Technical  Report  No. 
1078).  Alexandria,  VA:  U.S.  Army  Research  Institute  for  the  Behavioral  and  Social  Sciences. 
(ADA343575) 

Roberts,  D.F.,  Foehr,  U.G.,  and  Rideout,  V.  (2005).  Generation  M:  Media  in  the  lives  of  8  -  18  year-olds 
(Report  No.  7251).  Menlo  Park,  CA:  The  Henry  J.  Kaiser  Family  Foundation. 

Salter,  M.  S.,  Eakin,  D.  E.,  &  Knerr,  B.  W.  (1999).  Dismounted  warrior  network  enhancements  for  restricted 
terrain  (DWN  ERT):  An  independent  assessment  (Research  Report  No.  1742).  Alexandria,  VA:  U.S. 
Army  Research  Institute  for  the  Behavioral  and  Social  Sciences.  (ADA364607 

Singer,  M.J.,  Allen,  R.C.,  McDonald,  D.P.,  &  Gildea,  J.P.  (1997).  Terrain  appreciation  in  virtual 
environments:  Spatial  knowledge  acquisition  (Technical  Report  1056).  Alexandria,  VA:  U.S.  Army 
Research  Institute  for  the  Behavioral  and  Social  Sciences.  (ADA325520) 


RTO-MP-HFM-1 36 


21  -11 


Current  Issues  in  the  Use  of 

Virtual  Simulations  for  Dismounted  Soldier  Training 


21  -12 


RTO-MP-HFM-136 


Current  Issues  in  the  Use  of 
Virtual  Simulations  for 
Dismounted  Soldier  Training 


Bruce  W.  Knerr 
US  Army  Research  Institute 


Our  Research  Goal 


Develop  and  evaluate  technologies  and  methods 
to  train  dismounted  Soldiers  effectively  at 


reasonable  cost 


The  Current  Approach 
•Labor  Intensive 
•Time  consuming 
•Inflexible 


* 

Ik  R 

i 

The  Issues 


Our  research  has  focused  on  the  use  of  high 
fidelity,  immersive  simulation  to  supplement  live 
training. 

Today  there  are  low-cost  alternatives: 

-  Reduced  fidelity  interface  using  desktop  PCs 

-  Game-based  simulations  (either  commerical  or 
especially  developed) 

How  do  these  compare  in  terms  effectiveness? 


An  Illustration:  The  Virtual-Integrated 
MOUT  Training  System  (V-IMTS) 


Goals 

•  Gather  lessons  learned  and  obtain  Soldier  feedback  for 
future  systems 

•  Evaluate  perceived  training  effectiveness 


V-IMTS  Network  Configuration 
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Soldier  Visualization  Station  (SVS) 

Immersive  System 

•Rear  Screen  Projection 
•Tracks  Head  and  Weapon 
Position 

•Weapon-mounted  Joystick 
used  for  Locomotion 
•Simulated  Radio 
Communications 
•Array  of  Weapon  Types 
•Fragmentation, 

Concussion,  &  Smoke 
Grenades 
•Wounding 

Desktop  System 

•Same  capabilities  with  monitor  and  joystick 


Virtual 

Exercises 


Ratings  of  Skill  Improvement  as  a  Result  of 

Virtual-Live  Training 


Overall 

Communicate  with  team  or  squad. 
Coordinate  with  chain  of  command 
Control  squad  or  team 
Plan  tactical  operation 
Control  movement  not  in  contact 

Clear  building 
Clear  room 
Locate  enemy  positions 
Control  movement  assault 
Assess  tactical  situation 
React  to  Contact 


o 

None 


1 

Slight 


2 

Moderate 


3 

Vast 


Performing  Combat  Activities  in 

Virtual  Simulators 


•  Activities  Leaders  and  Soldiers  said  they  could  perform  well 
include  outdoor  movement  and  visual  identification  of  types  of 
people  and  significant  terrain. 

-  Examples 

•  Move  through  open  areas  as  a  widely-separated  group 

•  Identify  non-combatants  within  a  room 

•  Execute  planned  route 

•  Locate  assigned  areas  of  observation 

•  Activities  Leaders  and  Soldiers  said  they  could  not  perform 
well  include  indoor  (precise)  movement  and  sound 
localization. 

-  Examples 

•  Determine  the  direction  enemy  rounds  are  coming  from 

•  Climb  up  or  down  stairs 

•  Distinguish  between  friendly  and  enemy  fire 

•  Look  around  corners 

•  Soldier  (Desktop)  ratings  non-significantly  higher  than  Leader 
(Immersive)  ratings  (1.76  vs.  1.66) 

•  Correlation  between  Leader  and  Soldier  ratings  was  +.62 


Immersive  Simulator  Cost 
Elements  (US  $K) 


Component 

Low 

Estimate 

High 

Estimate 

Tracking  System 

43 

50 

HMD  (1280X1024) 

26 

40 

Projection  Display 

30 

350 

Targeting/Augmented  Display 

5 

8 

Binoculars/Night  Vision  Goggles 

4 

25 

Sound  System 

3 

8 

Total  with  Projection  Display 

85 

441 

Total  with  HMD 

81 

131 

Source:  Loftin,  Dryer,  et  al  (2004) 


Does  Immersion  Facilitate 
Learnina  and  Transfer? 


•  Research  has  largely  addressed  the  question  of  whether 
immersive  or  desktop  VEs  can  be  effective,  and  has  not 
generally  compared  their  effectiveness. 

•  Exceptions 

-  Loftin,  Scerbo,  et  al  (2004)  found  small  benefit  for  immersive 
system  for  training  in  checkpoint  operations. 

-  Jaquet  (2002)  found  small  benefit  for  VR  over  a  desktop 
simulator  for  maintenance  training. 

-  Grant  and  Magee  (1997)  found  a  walking  interface  improved 
spatial  learning  relative  to  joystick  control  of  movement. 

-  Singer  et  al  (1997)  found  a  head-linked  HMD  and  treadmill 
produced  better  spatial  learning  than  HMD  and  joystick. 

•  It  appears  that  immersive  simulations  do  provide  an 
advantage  for  training  spatially  oriented  tasks,  but  this 
difference  is  not  large. 

•  Lack  of  convincing  evidence  that  immersive  simulations 
are  cost-effective  for  Soldier  training 


Are  Computer  Games  Effective? 


Commercial  games  are  increasingly  proposed  as  a 
training  solution 

-  Can  be  single-player,  multi-player,  or  massively  multi-player 

-  Underlying  assumption  that  Soldiers  are  enthusiastic  and 
proficient  gamers 

Are  they  effective  for  training  Soldier  skills? 

-  Researchers  agree  that  they  can  be 

-  Evidence  for  Soldier  and  Leader  training  effectiveness  limited  to 
subjective  data 
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Do  Soldiers  play  PC  and  video 

games? 

•  US  youth  8-18  years  old 
average  70  minutes  of  game 
playing  per  day. 

•  Small  samples  of  enlisted 
Solders  in  ARI  evaluations 
appear  similar,  but  7-22%  of 
enlisted  and  30  -  57%  of  junior 
officers  reported  not  playing. 

•  No  data  regarding  the  types  of  games  (strategy, 
simulation,  first-person  shooter)  that  they  play. 

•  Cannot  assume  that  all  trainees  are  proficient  “gamers.” 
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Summary 


•There  is  a  lack  of  evidence  based  on  objective  measures 
of  performance  which  would  lead  to  recommendations 
about  the  appropriate  roles  of  immersive  simulations,  low 
cost  simulations,  and  PC-based  games  for  training 
dismounted  Infantry  Soldiers,  leaders,  and  small  units. 

•Without  this  evidence,  the  Army  does  not  have  a  sound 
basis  for  selecting  the  most  cost  effective  mix  of  training 
methods. 

•Our  knowledge  of  trainee  proficiency  with  and  motivation 
to  use  computer  or  video  games  is  limited. 

•Game  playing  experience  has  implications  for  the  types  of 
games  that  could  be  used  for  training  and  the  amount  and 
type  of  prerequisite  game  playing  training  required. 


